Method and device for editing composite content file and reproduction apparatus

ABSTRACT

A composite content file editing method includes the steps of dividing each media data included in the composite content file on the time base so as to generate a first composite content file to be reproduced first on the time base and a second composite content file to be reproduced later, and adding, as synchronizing information, information about a position shift on the time base at reproduction start of each media data included in the second composite content file to the second composite content file.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and device for editing acomposite content file including a plurality of types of multiplexedmedia data, and to a reproducing apparatus for the composite contentfile.

2. Description of the Prior Art

Recently, a composite content file including a plurality of types ofmultiplexed media data such as video data, audio data or text data hasbeen used in a content delivery service or a streaming broadcastingdirected to mobile terminals. One of file formats of the compositecontent file is the MP4 file format (hereinafter referred to as “MP4”)that is defined in an article of Part 14 of the ISO/IEC 14496 standard.

The system layer of MP4 includes a plurality of mixed types of media(media data) and is provided with a header portion that storesinformation such as conditions for reproducing the media and a mediadata portion that stores only a media stream. In this system layer,individual media are stored in a packet so as to be multiplexed in thetemporal order. The header portion (moov box) that includes mediainformation as header information of a stored packet and the media dataportion (media data box) that includes media data itself are completelyseparated from each other. In this respect, MP4 is different from asystem layer such as MPEG-2 PS or TS.

FIG. 20 is a diagram showing an example of a conventional MP4 fileformat FT1.

As shown in FIG. 20, a file type box BXA of the MP4 file format FT1stores information indicating compatibility of the file. A moov box BXBthat is a header portion stores information about a reproductioncondition of each media data stored in a media data box BXC that will bedescribed later, which includes position information, time information,size information and the like of a media frame. The media data box BXCstores media data such as video data, audio data, text data or the like.

The MP4 file format includes the time information of each media frame,which is not reproduction time but a reproduction time length. In otherwords, the time information indicates, for example, that a first frameof the video data is reproduced for ◯◯ milliseconds and a second frameis reproduced for ΔΔ milliseconds. Therefore, video data is reproducedonly by the reproduction time length of the video data, while audio datais reproduced only by the reproduction time length of the audio data.

A user of a mobile terminal can receive delivery of such a compositecontent file of the MP4 file format by his or her mobile terminal andreproduce the file. However, a maximum size of content that a mobileterminal can handle depends on a type of the mobile terminal. Therefore,if a size of the content exceeds the maximum size of content that themobile terminal can handle, a server that delivers the content isrequired to divide the content into a plurality of files (compositecontent files), while the mobile terminal is required to reproduce theplurality of files continuously.

As a device that divides an MMS message having a size above atransmission capacity of a server into files having a size that themobile terminal can transmit, there is proposed a device that isdescribed in U.S. patent application publication No. 2005/0054287. Thedevice disclosed in the publication includes a receiving portion thatreceives an input such as an image signal, an audio signal and the like,a control portion that controls individual portions of the mobileterminal and encodes the image signal and the audio signal received viathe receiving portion into multimedia data, which is divided into aspecific size and stored as the divided data in an designated order, abuffer that stores the multimedia data and the divided data asindividual files, a memory portion that stores the individual filesstored in the buffer by the control portion in corresponding areas inaccordance with the order, an output portion that delivers operationalinformation of the mobile terminal, the image signal or the audio signalin accordance with the control portion, and a radio frequency portionthat transmits the files stored in the memory portion by wireless.

However, in the conventional method, since the synchronizing informationof each media data is not stored as information about the reproductioncondition of each of the divided media data, there may be a problem asfollows.

FIGS. 21 and 22 are diagrams showing examples of the method for dividingthe media data.

Positions on a time base that divide each media data included in thecontent (hereinafter referred to as “division points”) are usually basedon the video data and are usually positions of boundaries betweenpictures of the video data as shown in FIG. 21. The reason is thatdivision of the video data should be performed so that an I-pictureframe that can be reproduced by itself becomes a head of the video dataafter division, and therefore division points depend on positions of theI-picture necessarily.

In this case, therefore, if a filing section of an elementary stream isdesignated for example, it is necessary to include data of thedesignated section completely. As a result, a section of each media hasa range little wider than that including it.

However, there is very little possibility that the division point of thevideo data decided as described above matches a boundary between audioframes completely. Therefore, as shown in FIG. 21, the audio data isdivided at a boundary between frames that is closest to the positioncorresponding to the division point of the video data.

In this case, when the individual media data divided as described aboveare reproduced, if heads of media data at the reproduction start arejustified, a reproduction timing of the audio data is delayed from thatof the video data by shift time T1 of the division point in a secondfile. In addition, reproduction end timings of the individual media dataare shifted in a first file and the second file.

Such a shift of timing causes an uncomfortable feeling that the user mayhave, which includes a situation that a motion of the picture does notmatch the sound, a situation that the sound is still ringing even whenthe picture is finished, a situation that a sound is interrupted, andthe like.

In addition, a method may be considered for matching positions on a timebase between the video data and the audio data when they are reproduced,in which each media data is divided at boundaries between audio framesas shown in FIG. 22.

In this case, however, the video data is to be divided in the I-pictureframe. Then, since each of the two divided I-picture frames has to bereproducible by itself, a complete I-picture frame is to be used foreach of them.

Therefore, when the division as described above is performed, each ofthe first and the second files includes the I-picture having a largesize so that reproduction time of content included in one file becomesshort.

SUMMARY OF THE INVENTION

An object of the present invention is to provide method and device forediting a composite content file, which do not make a user feel a shiftbetween a picture and a sound when they are reproduced after thecomposite content file is divided.

A method according to one aspect of the present invention includes thesteps of dividing each media data included in the composite content fileon the time base so as to generate a first composite content file to bereproduced first on the time base and a second composite content file tobe reproduced later, and adding, as synchronizing information,information about a position shift on the time base at reproductionstart of each media data included in the second composite content fileto the second composite content file.

Preferably, information about a position shift on the time base atreproduction end of each media data included in the first compositecontent file may be added to the first composite content file as thesynchronizing information.

Since the synchronizing information is provided, a shift between apicture and a sound can be cancelled by performing correction duringreproduction based on the synchronizing information.

A reproducing apparatus according to one aspect of the present inventionincludes a synchronizing information obtaining portion that obtainssynchronizing information that is added to the composite content file, aposition shift information extracting portion that extracts informationabout a position shift on the time base during reproduction of eachmedia data from the synchronizing information, a file informationextracting portion that extracts file information of a composite contentfile to be reproduced continuously next to the composite content filefrom the synchronizing information, a correction portion that correctsreproduction time during reproduction of each media data based on theextracted information about a position shift on the time base, and amedia data obtaining portion that reads out the next composite contentfile based on the extracted file information and obtains media data thatsucceeds each media data that has been reproduced among the individualmedia data included in the composite content file that is beingreproduced so as to reproduce the media data continuously when thereproduction of the composite content file that is being reproduced isfinished.

If file information of the composite content file to be reproduced nextcontinuously is extracted, the next composite content file is read outbased on the file information so that the continuous reproduction isperformed.

According to the present invention, it is possible that a user does notfeel a shift between a picture and a sound when a composite content fileis divided and reproduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an example of a general structure of acontent delivering system.

FIG. 2 is a block diagram showing an example of a hardware structure ofa content editing apparatus.

FIG. 3 is a block diagram showing an example of a hardware structure ofa mobile terminal.

FIG. 4 is a block diagram showing a functional structure of the contentediting apparatus.

FIG. 5 is a block diagram showing a structure of a reproducing apparatusin single reproduction.

FIG. 6 is a block diagram showing a structure of the reproducingapparatus in continuous reproduction.

FIG. 7 is a block diagram showing a functional structure of the contentediting apparatus.

FIG. 8 is a diagram showing an example of synchronizing information.

FIG. 9 is a diagram showing an example of a structure of a content file.

FIG. 10 is a diagram showing an example of division of media data of thecontent file.

FIG. 11 is a diagram showing an example of first media data after thedivision.

FIG. 12 is a diagram showing an example of second media data after thedivision.

FIG. 13 is a diagram showing an example of synchronizing information forthe content file after the division.

FIG. 14 is a diagram showing a functional structure of the mobileterminal about reproduction of content.

FIG. 15 is a diagram showing an example of correction in the continuousreproduction.

FIG. 16 is a diagram showing an example of correction in the singlereproduction of the first content file.

FIG. 17 is a diagram showing an example of correction in the singlereproduction of the second content file.

FIG. 18 is a flowchart showing a general flow of file editing.

FIG. 19 is a flowchart showing a general flow of content reproduction.

FIG. 20 is a diagram showing an example of a conventional MP4 fileformat.

FIG. 21 is a diagram showing an example of a method for dividing themedia data.

FIG. 22 is a diagram showing an example of a method for dividing themedia data.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will now be described in detail with reference to theattached drawings.

In FIG. 1, the content delivering system 1 includes a content deliveringserver 11, a media controlling server 12, the content editing apparatus13 and the like.

The content delivering server 11 is connected to a mobile communicationnetwork (not shown) including gateways, exchanges, a base stationcontrol unit, base stations and the like. The content delivering server11 delivers content responding to a request from the mobile terminal UE.In other words, the content delivering server 11 requests the mediacontrolling server 12 to send the content requested by the mobileterminal UE, and it performs a process for transmitting the contentreceived from the media controlling server 12 to the mobile terminal UE.

The media controlling server 12 manages many content CT and transmitsdesignated content CT to the content delivering server 11 responding toa request from the content delivering server 11. The media controllingserver 12 can store and manage various media and various formats of dataas the content CT. The content CT includes a composite content file inwhich a plurality of types of media data are multiplexed, for example.The composite content file is made up of a combination of differenttypes of media data including video data, audio data, text data and thelike.

The media controlling server 12 encodes the content CT into a formatthat is suitable for delivery, if necessary, or converts the same into afile of a format that is suitable for delivery. In addition, if a sizeof the content CT or the file is too large to be delivered to the mobileterminal UE that made the request, the media controlling server 12requests the content editing apparatus 13 to divide it into a pluralityof files and to edit the same.

The content editing apparatus 13 generates (edits) two or more contentfiles (composite content files) CFB from the content CT responding tothe request from the media controlling server 12, and it sends thegenerated content files CFB to the media controlling server 12.

In other words, for example, the content editing apparatus 13 divideseach of the media data included in the content file CF that includescontent CT on time base, generates a first content file (first compositecontent file) to be reproduced first on the time base and a secondcontent file (second composite content file) to be reproduced later, andadds information about a position shift on the time base at thereproduction start of each media data included in the second contentfile to the second content file as synchronizing information.

Furthermore in this case, information about a position shift on the timebase at the reproduction end of each media data included in the firstcontent file is added to the first content file as the synchronizinginformation. Further, file information of the second content file isadded to the first content file as the synchronizing information.Further, information about a position shift on the time base at thereproduction end of each media data included in the second content fileis added to the second content file as the synchronizing information.

The content editing apparatus 13 divides the content file CF so that asize thereof (file capacity) is reduced to a size that can be receivedand reproduced by the mobile terminal UE. For example, if a size of afile that can be handled by the mobile terminal UE is 500 kilobytes anda size of a content file CF having a file name “A” shown in FIG. 1 is 1megabyte, the mobile terminal UE cannot receive the content file CF aslong as it is a single file. Therefore, in this case, the contentediting apparatus 13 divides the content file CF into two, for example,so as to generate two content files CFB each of which has 500 kilobytes,for example.

In addition, the content editing apparatus 13 adds the synchronizinginformation to the divided content files CFB when the content file CF isdivided. Since the synchronizing information is added, a time shift isnot generated between a picture by the video data and a sound by theaudio data and a character display by the text data when the dividedcontent file CFB is reproduced by the mobile terminal UE, so that a userof the mobile terminal UE does not feel a shift between them. Inaddition, a single content file CFB can be reproduced by itself, or aplurality of content files CFB can be reproduced continuously. There isno time shift between a picture and a sound, for example, in any case.

Note that when the synchronizing information is added to the contentfile CFB, it is possible to combine the individual divided media dataand the generated synchronizing information so as to generate a singlecontent file (composite content file) each, similarly. Details thereofwill be described later.

As shown in FIG. 2, the content editing apparatus 13 is made up of a CPU21, a RAM 22, a ROM 23, a magnetic storage device 24, a communicationinterface 25 and the like. It may also be equipped with an output devicesuch as a display device, a printer or the like, and an inputoperational device such as a keyboard and a pointing device.

As shown in FIG. 3, the mobile terminal UE is made up of a CPU 31, aflash memory 32, a ROM 33, a communication unit 34, a display 35, aspeaker (or a earphone) 36, an operational button 37 and the like. Itmay be equipped with a DSP, camera and the like.

Next, an edit process of the content file CF in the content editingapparatus 13 and a reproduction process of the content file CFB in themobile terminal UE will be described.

FIG. 4 is a block diagram showing a functional structure of the contentediting apparatus 13, FIGS. 5 and 6 are block diagrams showing astructure concerning reproducing apparatuses 50 and 50B of the contentfile CF in the mobile terminal UE. Furthermore, FIG. 5 shows the casewhere the content file CF is reproduced by itself, while FIG. 6 showsthe case where a plurality of content files CF are reproducedcontinuously.

In FIG. 4, the content editing apparatus 13 includes a video encoderportion 41, an audio encoder portion 42, a text encoder portion 43, aheader generating portion 44 and a packing portion 45.

Individual media data DM included in the content file CF, i.e., videodata DV, audio data DU and text data DT are entered to the contentediting apparatus 13. A set of the video data DV, the audio data DU andthe text data DT may be referred to as “media data group.” Note that itis possible to provide a memory for storing the content file CF or theindividual media data DM thereof temporarily in the content editingapparatus 13.

The video encoder portion 41 encodes the entered video data DV into adesired and appropriate format. The audio encoder portion 42 encodes theentered audio data DU into a desired and appropriate format. The textencoder portion 43 encodes the entered text data DT into a desired andappropriate format.

The header generating portion 44 is provided with a synchronizinginformation generating portion 46 that generates synchronizinginformation such as time information, position information or the likefor each of the encoded media data when it is divided. The synchronizinginformation and the reproduction condition information generated in theheader generating portion 44 are stored as a header portion in anappropriate memory area of the header generating portion 44.

The packing portion 45 crams media data delivered from the individualencoder portions 41-43 and a header portion delivered from the headergenerating portion 44 into one or more composite content files so as tobe packed. In other words, the packing portion 45 packs the plurality ofencoded media data and the header portion and generates one or morecomposite content files having a predetermined format such as an MP4file format. The generated composite content files are delivered ascontent files CFB1, CFB2 and so on.

In FIG. 5, the reproducing apparatus 50 is made up of a buffer 51, afile analyzing portion 52, a synchronous correction portion 53, a headermanaging portion 54, a display processing portion 55 and the like.

The buffer 51 is a memory area for storing a composite content file thatis downloaded to the mobile terminal UE.

The file analyzing portion 52 analyzes the composite content file thatis downloaded to the buffer 51, and delivers the header portion thereofto the header managing portion 54. In addition, if the header portionincludes the synchronizing information, the synchronous correctionportion 53 corrects the reproduction time based on the synchronizinginformation, so that a time shift is not generated between the media ofvideo and audio or the like.

The header managing portion 54 stores and manages the codec information,the time information, the size information, offset information and thelike included in the entered header portion.

The display processing portion 55 refers to sample information that ismanaged in the header managing portion 54, decodes the entered mediadata with a decoding portion 56, and displays video and text on thedisplay 35 based on the information in a time information managingportion 57.

In FIG. 6, the reproducing apparatus 50B is made up of two buffers,i.e., an A-buffer 51A and a B-buffer 51B, the file analyzing portion 52,the synchronous correction portion 53, the header managing portion 54,the display processing portion 55, a continuous file requesting portion58, an audio processing portion 59 and the like.

As to the reproducing apparatus 50B shown in FIG. 6, parts having thesame functions as the reproducing apparatus 50 shown in FIG. 5 aredenoted by the same references so that description thereof will beomitted.

The A-buffer 51A and the B-buffer 51B are memory areas for storing aplurality of content files CFB1 and CFB2 alternately that are downloadedto the mobile terminal UE.

The file analyzing portion 52 analyzes the content file CFB1 that isdownloaded to the A-buffer 51A and delivers a header portion thereof tothe header managing portion 54. Then, if the synchronizing informationincludes continuous file information, it delivers the continuous fileinformation to the continuous file requesting portion 58.

Note that the “continuous file information” corresponds to the “fileinformation” of the present invention. In addition, the “continuous fileinformation” is also information for identifying a media data group tobe reproduced continuously.

In addition, the media data is delivered to the display processingportion 55 and to the audio processing portion 59. Using data of theheader managing portion 54, a decoding process and a time managingprocess are performed, which are displayed on the display 35 or a soundis produced from the speaker 36. In this case, if the header portionincludes the synchronizing information, the synchronous correctionportion 53 performs a process of correcting the head and the lastreproduction time information of a video frame in accordance with acorrection value. Then the same process is performed for the contentfile CFB2 that is downloaded to the B-buffer 51B.

After confirming that the first content file CFB has been downloaded,the header managing portion 54 instructs the continuous file requestingportion 58 to download the next content file CFB. Note that a headerA-managing portion 54A of the header managing portion 54 manages headerinformation of the content file CFB1 that is downloaded to the A-buffer51A while a header B-managing portion 54B manages header information ofthe content file CFB2 that is downloaded to the B-buffer 51B.

The continuous file requesting portion 58 refers to the continuous fileinformation delivered from the file analyzing portion 52 and performs aprocess of requesting the content delivering server 11 to deliver thenext content file CFB that succeeds the content file CFB that is nowbeing processed.

Next, the content editing will be described further in detail.

FIG. 7 is a block diagram showing a functional structure of the contentediting apparatus 13 about the content editing, FIG. 8 is a diagramshowing an example of the synchronizing information, and FIG. 9 is adiagram showing an example of a structure of the content file CFB thatis generated by the content editing apparatus 13.

In FIG. 7, the content editing apparatus 13 includes a dividing portion61, a reproduction condition information generating portion 62 and asynchronizing information generating portion 63.

The dividing portion 61 divides each media data included in the contentfile CF as it is or after appropriate encoding, on the time base, so asto generate first media data (a first media data group) DMA that shouldbe reproduced first on the time base and second media data (a secondmedia data group) DMB that should be reproduced later.

The reproduction condition information generating portion 62 generatesreproduction condition information RT that is information forreproducing each of the media data DM. The reproduction conditioninformation RT indicates time information that indicates reproductiontime for each frame of each media data DM, size information thatindicates a size of each frame of each media data DM, offset informationthat indicates a place where each frame of each media data DM is stored,chunk information that indicates a unit of chunk of frames, and thelike. The reproduction condition information RT and the synchronizinginformation DT described above constitute a so-called header of eachmedia data DM.

The synchronizing information generating portion 63 generatessynchronizing information DT that is the information about a positionshift on the time base when each media data DM is reproduced.

As shown in FIG. 8, the synchronizing information DT includes asynchronizing information tag indicating to be synchronizinginformation, a synchronizing information size that indicates a size ofthe synchronizing information DT, continuous file information DTF thatis information about the composite content file to be reproducedcontinuously after the composite content file related to thesynchronizing information DT, head picture correction information DTTthat is the information about a position shift on the time base at thereproduction start of each media data DM, later picture correctioninformation DTB that is the information about a position shift on thetime base at the reproduction end of each media data DM, and the like.

In addition, as described above, the packing portion 45 edits (packs)the encoded media data DM, the reproduction condition information RT andthe synchronizing information DT to be a single file, so that thecontent file CFB is generated. In other words, the packing portion 45adds the synchronizing information DT to the media data DM.

In FIG. 9, the content file CFB is made up of a file type box BXA, amoov box BXB, a media data box BXC and a synchronizing box BXD. Sincesome of them have the same functions as the boxes that are describedabove with reference to FIG. 21, different functions will be describedmainly.

The moov box BXB is made up of storage areas stbl for storing thereproduction condition information RT of the individual media data DM.More specifically, each of the storage areas stbl includes an area sttsfor storing time information that indicates the reproduction time ofeach frame of each media data DM, an area stsz for storing sizeinformation that indicates a size of each frame of each media data DM,an area stsc for storing offset information that indicates a place forstoring each frame of each media data DM, and an area stco for storingchunk information that indicates a unit of chunk of frames.

The synchronizing box BXD stores the synchronizing information DTgenerated by the synchronizing information generating portion 63. Notethat the synchronizing box BXD is newly defined and provided in thepresent embodiment as an extension area of the MP4.

Next, a concrete example of the file editing (composite content fileediting) will be described.

FIG. 10 is a diagram showing an example of division of the media data DMof the content file CF, FIG. 11 is a diagram showing an example of firstmedia data DMA after the division, FIG. 12 is a diagram showing anexample of second media data DMB after the division, FIG. 13 is adiagram showing an example of synchronizing information DT1 and DT2about the content file CFB after the division. Note that the first mediadata DMA constitutes the first content file CFB1 while the second mediadata DMB constitutes the second content file CFB2.

In FIG. 10, a division point of the video data is determined first.Then, a division point of the audio data is determined to be a boundarybetween frames that is closest to a position corresponding to thedivision point of the video data. A division point of the text data isdetermined to be a position corresponding to the division point of thevideo data. Thus, division points of each media data are determined, andeach media data DM are divided at each of the division points.

Note that although FIG. 10 shows the example where each media data DM isdivided into two, it is possible to divide each media data DM into threeor more. The number of division should be determined so that the contentfile CFB after the division becomes a size that can be delivered to themobile terminal UE.

In addition, the head picture correction information TDT and the laterpicture correction information BDT (see FIG. 8) that are thesynchronizing information DT are generated by calculating the positionshift on the time base at the reproduction start and at the reproductionend of the video data and the audio data.

For example, when each media data DM is divided as shown in FIG. 10, itis necessary to pay attention first to the position shift on the timebase between the video data and the audio data that should be reproducedfirst on the time base at the reproduction end.

In this case, as shown in FIG. 11, the position on the time base of thevideo data at the reproduction end is longer than (delayed from) that ofthe audio data by 10 milliseconds. For this reason, reproduction of theP-picture of the video data at the reproduction end should be finishedearlier by 10 milliseconds for synchronization between the video dataand the audio data at the reproduction end.

Therefore, the later picture correction information DTB1 of thesynchronizing information DT1 of the first content file CFB1 becomes“−10 milliseconds” (see FIG. 13). As to the head picture correctioninformation DTT1, since there is no position shift on the time basebetween the video data and the audio data at the reproduction start, itbecomes “0”.

As to the video data and the audio data that are stored in the secondcontent file CFB2, as shown in FIG. 12, the position on the time base ofthe video data at the reproduction start is shorter than (delayed from)that of the audio data by 10 milliseconds. In addition, the position onthe time base of the video data at the reproduction end is shorter(earlier) than that of the audio data by 6 milliseconds. For thisreason, it is necessary to extend the reproduction time of the I-pictureof the video data at the reproduction start by 10 milliseconds and toextend also the reproduction time of the P-picture of the video data atthe reproduction end by 6 milliseconds, for synchronization between thevideo data and the audio data at the reproduction. Therefore, the headpicture correction information DTT2 of the synchronizing information DT2of the second content file CFB2 becomes “10 milliseconds” and the laterpicture correction information DTB2 becomes “6 milliseconds” (see FIG.13).

As to the continuous file information DTF1 of the first content fileCFB1, a URL (Uniform Resource Locator) that indicates a location forstoring the second content file CFB2 or the like can be used. Inaddition, if there is no file to be reproduced next continuously likethe second content file CFB2, the continuous file information DTF2becomes “Null”.

Next, content reproduction (composite content file reproduction) will bedescribed further in detail.

FIG. 14 is a diagram showing a functional structure of the mobileterminal UE about the content reproduction.

In FIG. 14, the mobile terminal UE includes a synchronizing informationobtaining portion 71, a position shift information extracting portion72, a file information extracting portion 73, a media data obtainingportion 74, a reproduction control portion 75, a correcting portion 76and the like.

The synchronizing information obtaining portion 71 obtains thesynchronizing information DT that is added to the content file CFBdownloaded to the A-buffer 51A or the B-buffer 51B.

The position shift information extracting portion 72 extractsinformation about a position shift on the time base (the head picturecorrection information DTT and the later picture correction informationDTB) of each media data DM during the reproduction from thesynchronizing information DT.

The file information extracting portion 73 extracts the continuous fileinformation DTF that is file information of the content file CFB to bereproduced continuously after the content file CFB from thesynchronizing information DT.

The media data obtaining portion 74 reads out the next composite contentfile based on the extracted continuous file information DTF and obtainsthe media data DM that succeeds it for each media data DM that has beenreproduced among the individual media data DM included in the contentfile CFB that is being reproduced, so that it is reproduced continuouslywhen the reproduction of the content file CFB that is being reproducedis finished.

The reproduction control portion 75 performs a process and a control forreproducing the media data DM of the content file CFB.

The correcting portion 76 corrects the reproduction time of the eachmedia data DM during the reproduction based on the extracted informationabout a position shift on the time base. When the correction isperformed, if no continuous file information DTF is extracted from thesynchronizing information DT, correction of the reproduction time at thereproduction end of each media data DM included in the content file CFBis performed. In other words, if the continuous file information DTF isextracted from the synchronizing information DT, the correction is notperformed when the content file CFB is reproduced.

Next, a concrete example of the content reproduction will be described.First, the continuous reproduction will be described.

FIG. 15 is a diagram showing an example of the correction in thecontinuous reproduction.

In FIG. 15, since the synchronizing information DT1 of the first contentfile CFB1 includes the continuous file information DTF1 (see FIG. 13),the second content file CFB2 to be reproduced next continuously isreproduced continuously. In other words, for each media data DM that hasbeen reproduced among the individual media data DM of the first contentfile CFB1, end of the other media data DM is not waited, but the mediadata DM to be reproduced next is read out from the buffer and isreproduced continuously without an interval.

In the example shown in FIG. 15, just after the last P-picture of thevideo data DV of the first content file CFB1 is reproduced for a regulartime period, the first I-picture of the second content file CFB2 isreproduced for a regular time period in the same manner. As to the audiodata DU and the text data DT, the first frame of the second content fileCFB2 is reproduced following the last frame of the first content fileCFB1.

When the second content file CFB2 is reproduced, the synchronizinginformation DT2 checks whether or not there is a content file CFB to bereproduced next. Since there is no continuous file information DTF2 (seeFIG. 13), it is recognized that the second content file CFB2 is thelast. In this case, the correction is performed, if necessary, only atthe reproduction end of the second content file CFB2.

In this way, the content files CFB1 and CFB2 divided by the contentediting apparatus 13 are reproduced in the mobile terminal UE so thatthe individual media data DM can be reproduced continuously, and thecorrection is performed, if necessary, at the reproduction end. As aresult, a user of the mobile terminal UE does not feel any changecompared to the case where the original content file CF is reproduced atit is. In addition, even if time lengths of the individual media data DMare different in the original content file CF, incongruity feeling iscanceled since the correction is performed when the reproduction isfinished.

Next, a case where the content files CFB1 and CFB2 are reproduced in asingle manner will be described.

FIG. 16 is a diagram showing an example of the correction of the firstcontent file CFB1 in the single reproduction, and FIG. 17 is a diagramshowing an example of the correction of the second content file CFB2 inthe single reproduction.

In FIG. 16, as to the single reproduction of the first content fileCFB1, a process of correcting a position shift on the time base of eachmedia data DM is performed based on the synchronizing information DT1.According to the synchronizing information DT1, the head picturecorrection information DTT1 is “0” and the later picture correctioninformation DTB1 is “−10 milliseconds” (see FIG. 13). Therefore, thecorrection is not performed at the reproduction start of the video dataDV. At the reproduction end thereof, the reproduction time of theP-picture is corrected to be shortened from 66 milliseconds to 56milliseconds. For this purpose, a reproduction speed is increased sothat the reproduction of the last P-picture can be finished in 56milliseconds, for example. Thus, shifts of position on the time baseamong the video data DV, the audio data DU and the text data DT at thereproduction end can be cancelled, so that synchronization can be taken.

In FIG. 17, according to the synchronizing information DT2, the headpicture correction information DTT2 is “10 milliseconds” and the laterpicture correction information DTB2 is “6 milliseconds”. Therefore, thereproduction time of the I-picture at the reproduction start of thevideo data is corrected from 66 milliseconds to 76 milliseconds, and thereproduction time of the P-picture at the reproduction end is correctedfrom 66 milliseconds to 72 milliseconds. For this purpose, thereproduction of the first I-picture is started at the same time as theaudio data DU, and a reproduction speed is decreased so that thereproduction time becomes 76 milliseconds, for example. In addition, areproduction speed of the last P-picture is decreased so that thereproduction time becomes 72 milliseconds.

In this way, if the divided content file CFB is reproduced in a singlemanner, correction of the position shift on the time base is performedso that there is no shift among all the media data DM at thereproduction start and the reproduction end without any shift. Thus, auser does not feel a shift between a picture and a sound when they arereproduced.

Note that when the divided content files CFB are reproduced by themobile terminal UE, a series of content files CFB1, CFB2, CFB3 and so onare usually reproduced continuously and sequentially. In this case, thecorrection is performed, if necessary, so that positions on the timebase match to each other at the reproduction end of the last contentfile CFB.

If a user wants to reproduce the divided content file CFB in a singlemanner, a button (not shown) that is provided to the mobile terminal UEis operated so that a “single reproduction mode” is selected, and thesingle reproduction as described above is performed. Further in thiscase, it is preferable to provide a button for selecting a content fileto be reproduced in a single manner from the divided plurality ofcontent files CFB1, CFB2, CFB3 and so on.

As described above, according to the file editing and contentreproducing method of the present embodiment, when any of the singlereproduction and the continuous reproduction of the content CT isperformed by the mobile terminal UE, a user can watch and listen to thecontent CT without incongruity feeling since synchronization betweenthem is not lost.

Further, since the synchronizing information DT that is provided forsynchronization has a small capacity like a few bytes, a process load isscarcely increased by providing the synchronizing information DT. It canbe realized easily by adding such a function to the conventional contentdelivering system 1.

Next, the file editing and the content reproduction will be describedwith reference to flowcharts.

FIG. 18 is a flowchart showing a general flow of the file editing, andFIG. 19 is a flowchart showing a general flow of the contentreproduction.

In FIG. 18, when the content CT or the content file CF to be edited issupplied (#11), encoding is performed, if necessary (#12). Each mediadata is divided on the time base so that a plurality of media datagroups are generated (#13). Information about a position shift on thetime base of each media data and information for identifying a mediadata group to be reproduced next continuously are generated as thesynchronizing information for each media data group (#14). In addition,the reproduction condition information RT is generated (#15). Individualmedia data groups and the synchronizing information for individual mediadata groups are combined so as to generate a plurality of content filesCFB (#16).

In FIG. 19, when the content file CFB is supplied (#21), file analysisis performed so that the synchronizing information DT is obtained (#22).If there is information about a position shift on the time base duringreproduction of each media data among the synchronizing information DT,it is extracted (#23). It is checked whether or not there is continuousfile information DTF, and it is obtained if there is (#24). If there iscontinuous file information DTF, the continuous reproduction isperformed (#26) in the case where it is not the single reproduction mode(No in #25) so that necessary correction is performed only at thereproduction end (#27).

If there is no continuous file information DTF (No in #24), or it is thesingle reproduction mode (Yes in #25), the reproduction is performedwhile correcting the reproduction time of each media data based on theextracted information about a position shift on the time base (#28).

In the embodiment described above, the structure of the whole or eachpart of the content editing apparatus 13, the reproducing apparatus 50,the content delivering system 1 or the mobile terminal UE, the processcontents, the process order, the structures of the content files CF andCFB, the structure of the synchronizing information DT and the like canbe modified if necessary in accordance with the spirit of the presentinvention.

While example embodiments of the present invention have been shown anddescribed, it will be understood that the present invention is notlimited thereto, and that various changes and modifications may be madeby those skilled in the art without departing from the scope of theinvention as set forth in the appended claims and their equivalents.

1. A composite content file editing method for editing a compositecontent file in which a plurality of types of media data aremultiplexed, the composite content file having a header portion in whichreproduction conditions including time information indicatingreproduction time lengths for each type of the media data are stored andalso a media data portion provided separately from the header portionand storing therein the media data, the method comprising: dividing eachmedia data included in the composite content file on the time base so asto generate a first composite content file to be reproduced first on thetime base and a second composite content file to be reproduced later,each of said composite content files comprising a group of media data;adding to the first composite content file, as synchronizinginformation, information about a position shift on the time base atreproduction start and reproduction end of video data that is one typeof said group of media data included in the first composite contentfile, and file information of the second composite content file; andadding to the second composite content file, as synchronizinginformation, information about a position shift on the time base atreproduction start and reproduction end of video data that is one typeof said group of media data included in the second composite contentfile.
 2. A composite content file editing apparatus for editing acomposite content file in which a plurality of types of media data aremultiplexed, the composite content file having a header portion in whichreproduction conditions including time information indicatingreproduction time lengths for each type of the media data are stored andalso a media data portion provided separately from the header portionand storing therein the media data, the apparatus comprising: a dividingportion that divides each media data included in the composite contentfile on the time base so that a first composite content file to bereproduced first on the time base and a second composite content file tobe reproduced later are generated, each of said composite content filescomprising a group of media data; and a synchronizing information addingportion that adds to the first composite content file, as synchronizinginformation, information about a position shift on the time base atreproduction start and reproduction end of video data that is one typeof said group of media data included in the first composite contentfile, and file information of the second composite content file, andadds to the second composite content file, as synchronizing information,information about a position shift on the time base at reproductionstart and reproduction end of video data that is one type of said groupof media data included in the second composite content file.